Display apparatus, information processing apparatus, and information processing method for displaying a second image that includes options for manipulating a first image

ABSTRACT

A projector includes a position detecting section that detects a pointing action, a projection section that projects handwritten characters on a screen based on the pointing action, and a UI controller that determines, based on the display direction of the displayed handwritten characters, the display direction of a UI screen that allows manipulation of the handwritten characters and projects the UI screen on the screen on the basis of the determined display direction.

CROSS-REFERENCE

The entire disclosure of Japanese Patent Application No. 2016-190778filed Sep. 29, 2016 is expressly incorporated by reference herein.

BACKGROUND 1. Technical Field

The present invention relates to a display apparatus, an informationprocessing apparatus, and an information processing method.

2. Related Art

In related art, to perform touch operation with a user's finger or anyother object on a display screen on which an object image is displayed,there is a proposed display apparatus that displays an operation pad,which is a user interface that accepts touch operation, in the vicinityof the object image, as shown in JP-A-2015-88090.

The display apparatus, when it displays the operation pad, determinesthe orientation of the displayed operation pad in such a way that theorientation of the operation pad coincides with that of the displayscreen having a rectangular shape. Therefore, when the user directlyfaces the display screen, the display apparatus displays the operationpad in such a way that the operation pad directly faces the user.

In a case where the user faces the object image in an oblique directionwith respect to the display screen, however, the user operates theoperation in the oblique direction pad because the operation pad isdisplayed in parallel to the edges of the rectangular shape of thedisplay screen, resulting in degradation in user friendliness anddecrease in operability.

SUMMARY

An advantage of some aspects of the invention is to provide a displayapparatus that excels in operability.

The invention can be implemented as the following forms or applicationexamples.

Application Example 1

A display apparatus according to this application example includes adetection section that detects a pointing action, a display section thatdisplays a first image based on the pointing action, a determinationsection that determines, based on a display direction of the first imagein the display section, a display aspect of a second image that allowsmanipulation of the first image, and a display control section thatcauses the display section to display the second image based on thedisplay aspect determined by the determination section.

According to the configuration described above, the display apparatusdisplays the first image based on a pointing action, determines thedisplay aspect of the second image, which allows manipulation of thefirst image, on the basis of the display direction in which the firstimage is displayed, and displays the second image on the basis of thedetermined display aspect. Therefore, since the display aspect of thesecond image is determined on the basis of the display direction of thefirst image, the visibility of the second image is improved, whereby theoperability of the second image can be improved.

Application Example 2

In the display apparatus according to the application example describedabove, it is preferable that the determination section determines thedisplay aspect of the second image in such a way that the displaydirection coincides with an orientation of the first image.

According to the configuration described above, since the second imageis so displayed that the orientation thereof coincides with the displaydirection of the first image, a user can readily issue a manipulationinstruction via the second image in the state in which the user directlyfaces the first image.

Application Example 3

In the display apparatus according to the application example describedabove, it is preferable that the display control section rotates thesecond image based on the display aspect and displays the rotated secondimage in a vicinity of the first image.

According to the configuration described above, since the second imageis so displayed in the vicinity of the first image that the orientationof the second image coincide with the display direction of the firstimage, the visibility of the second image corresponding to the firstimage is improved.

Application Example 4

In the display apparatus according to the application example describedabove, it is preferable that at least one of a displayed color,enlargement, reduction, and range specification of the second image iscapable of being specified with respect to the first image.

According to the configuration described above, at least one of thedisplayed color, enlargement, reduction, and range specification of thefirst image can be specified via the second image.

Application Example 5

In the display apparatus according to the application example describedabove, it is preferable that, in a case where the first image is formedof a plurality of first images and the plurality of first images aredisplayed in the display section, the determination section determinesthe display aspect of the second image based on the first image pointedby the pointing action.

According to the configuration described above, in the case where aplurality of first images are displayed, the display aspect of thesecond image can be determined on the basis of the first image pointedby the pointing action.

Application Example 6

In the display apparatus according to the application example describedabove, in a case where the pointing action points one of the pluralityof first images, the determination section may determine the displayaspect of the second image in such a way that an orientation of thesecond image coincides with the display direction of the pointed firstimage.

Application Example 7

In the display apparatus according to the application example describedabove, in a case where the pointing action points the plurality of firstimages, the determination section may determine the display aspect ofthe second image based on directions of the plurality of first images.

Application Example 8

An information processing apparatus according to this applicationexample is an information processing apparatus that processes a firstimage displayed based on a pointing action, the information processingapparatus including a determination section that determines, based on adisplay direction of the displayed first image, a display aspect of asecond image that allows manipulation of the first image and a displaycontrol section that displays the second image based on the displayaspect determined by the determination section.

According to the configuration described above, the informationprocessing apparatus determines the display aspect of the second image,which allows manipulation of the first image, on the basis of thedisplay direction in which the first image based on the pointing actionis displayed, and displays the second image on the basis of thedetermined display aspect. Therefore, since the display aspect of thesecond image is determined on the basis of the display direction of thefirst image, the visibility of the second image is improved, whereby theoperability of the second image can be improved.

Application Example 9

An information processing method according to this application exampleincludes detecting a pointing action, displaying a first image based onthe pointing action, determining, based on a display direction of thedisplayed first image, a display aspect of a second image that allowsmanipulation of the first image, and displaying the second image basedon the determined display aspect.

Since the method described above includes displaying the first imagebased on a detected pointing action, determining the display aspect ofthe second image, which allows manipulation of the first image, on thebasis of the display direction in which the first image is displayed,and displaying the second image on the basis of the determined displayaspect. Therefore, since the display aspect of the second image isdetermined on the basis of the display direction of the first image, thevisibility of the second image is improved, whereby the operability ofthe second image can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 shows the configuration of a projection system according toEmbodiment 1.

FIG. 2 is a functional block diagram showing functional portions thatform a projector.

FIG. 3 is a flowchart showing the procedure of a process carried outwhen a character drawing function is activated.

FIG. 4 shows a first UI screen projected on a screen.

FIG. 5 shows a second UI screen projected on the screen.

FIG. 6 shows a screen projected on the screen in a case where a rangeselection button is selected.

FIG. 7 shows an example in which the range of part of handwrittencharacters is specified.

FIG. 8 shows an instruction of enlargement of part of the handwrittencharacters.

FIG. 9 shows that the part of the handwritten characters is enlarged.

FIG. 10 shows an example of projection of a UI screen in a case where aplurality of sets of handwritten characters are drawn.

FIG. 11 shows an example in which an underline is drawn belowhandwritten characters.

FIG. 12 shows another example of the projection of a UI screen in a casewhere a plurality of sets of handwritten characters are drawn.

FIG. 13 shows an example in which a closed area containing a pluralityof sets of handwritten characters is drawn.

FIG. 14 shows the configuration of a projection system according toEmbodiment 2.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the invention will be described below with reference tothe drawings.

Embodiment 1

FIG. 1 shows the configuration of a projection system 1 according toEmbodiment 1 to which the invention is applied. In the presentEmbodiment 1, the projection system 1 includes a projector 10 as a kindof display apparatus.

In the present Embodiment 1, the projector 10 is a short-focal-lengthprojector that is installed immediately below a screen SC and projectsan image obliquely downward. The screen SC illustrated in the presentEmbodiment 1 is a flat plate or a curtain that is fixed to a wallsurface or stands on a floor surface. A wall surface can also be used asthe screen SC. In this case, the projector 10 may be attached to anupper portion of the wall surface used as the screen SC.

Instead, it is conceivable that the projector 10 is a table display,that is, an aspect in which the projector 10 is hung from the ceilingfor downward projection and projects an image on the screen SC installedon a horizontal surface, such as a table. Another aspect in which theprojector 10 is installed in a table and projects an image on a screenSC at the surface of the table from the rear of the screen SC isconceivable. An image projected by the projector 10 does not necessarilyhave a quadrangular shape defined with straight lines, and an aspect inwhich the projector 10 projects an image having a circular shape definedwith a curved line or a polygonal shape defined with straight lines isalso conceivable. For example, it is conceivable to employ an aspect inwhich the projection surface of the projector 10 hung from the ceilingis the surface of a circular table and users visually recognize an imageprojected in accordance with the shape of the circular table with theusers surrounding the projected image.

The projection system 1 provides an interactive function, that is, auser (operator) uses a pointing element 70 to instruct predeterminedoperation on the screen SC, on which the projector 10 projects an image.

The pointing element 70 is, for example, a pen-shaped input device andused in such a way that the operator holds a shaft 71 of the pointingelement 70 and presses the front end thereof against the screen SC. Anoperation switch 72, which detects the pressing operation, is providedat the front end of the pointing element 70, and when the operatorpresses the front end of the pointing element 70 against the screen SC,the operation switch 72 is turned on. The operator can perform positionpointing operation by pressing the front end of the pointing element 70against the screen SC and in an arbitrary position on the screen SC.

The projector 10 has the function of detecting the position of thepointing element 70 on the screen SC, as will be described later. Whenthe operator performs the position pointing operation, the projector 10detects the position where the front end of the pointing element 70comes into contact with the screen SC as an operation position.

The projector 10 accepts the position pointing operation performed bythe operator's pointing a position with the pointing element 70 and canreflect the operation in a projected image. Specifically, the projector10 can project, for example, handwritten characters 150, whichcorrespond to a first image, on the screen SC on the basis of the user'sdrawing action, that is, a pointing action in which the pointing element70 held by the user is moved with the pointing element 70 pressedagainst the screen SC.

For example, in FIG. 1, when the operator moves the pointing element 70relative to the screen SC from a lower left position toward an upperright position, the handwritten characters 150 are drawn on the screenSC in the oblique direction from the lower left position toward theupper right position.

When the front end of the pointing element 70 is pressed against thescreen SC, the projector 10 determines that a drawing action isinitiated. When the front end of the pointing element 70 is moved awayfrom the screen SC and a predetermined period elapses, the projector 10determines that the drawing action is completed.

The projector 10 can save an image drawn by the drawing action as imagedata.

FIG. 2 is a functional block diagram of portions that form the projector10.

The projector 10 includes an image data input section 11, to which imagedata is inputted.

The image data input section 11 may, for example, be a USB interface, ora communication interface, such as a wired or wireless LAN interface.For example, a server apparatus or another projector may be connected toa LAN interface provided in the image data input section 11.

The image data input section 11 may include an image input terminal towhich an analog video signal or digital image data is inputted. Theimage input terminal is, for example, a VGA terminal, a DVI (digitalvisual interface) terminal, an S video terminal, an RCA terminal, a Dterminal, an HDMI connector compliant with the HDMI (registeredtrademark) standard. The image data input section 11 may include aninterface circuit and an image processing circuit corresponding to theimage input terminals. The image data input section 11 may further havea reading function of reading image data stored, for example, in a flashmemory.

The projector 10 can be broadly divided into a projection section 20,which forms an optical image, and an image processing system thatprocesses image data.

The projection section 20 corresponds to a display section that displaysan image and includes an illumination system 21, a light modulator 22,and a projection system 23. The illumination system 21 includes a lightsource formed, for example, of a xenon lamp, an ultrahigh-pressuremercury lamp, an LED (light emitting diode), or a laser light source.The illumination system 21 may further include a reflector and anauxiliary reflector that guide light emitted from the light source tothe light modulator 22. Further, although not shown, the illuminationsystem 21 may still further include, for example, a lens group forenhancing optical characteristics of projection light, a polarizer, or alight adjusting element that is disposed in the path to the lightmodulator 22 and attenuates the amount of light emitted from the lightsource.

The light modulator 22 includes, for example, three transmissive liquidcrystal panels corresponding to the RGB three primary colors andmodulates light passing through the liquid crystal panels to generateimage light. The light from the illumination system 21 is separated intoRGB three color light fluxes, which are incident on the correspondingliquid crystal panels. The color light fluxes having been modulated bythe liquid crystal panels and having passed therethrough are combinedwith one another by alight combining system, such as a cross dichroicprism, and the combined light is outputted to the projection system 23.

The projection system 23 includes a zoom lens that enlarges and reducesan image to be projected and performs focal point adjustment, a zoomadjustment motor that adjusts the degree of zooming, a focus adjustmentmotor that adjusts focusing, a concave mirror that reflects projectionlight toward the screen SC, and other components. The projection system23 performs the zoom adjustment and focus adjustment on the image lightmodulated by the light modulator 22, guides the light having passedthrough the lens group toward the screen SC via the concave mirror, andforms an image on the screen SC.

A projection system driver 16 and a light source driver 14 are connectedto the projection section 20. The projection system driver 16 drives themotors provided in the projection system 23 under the control of acontrol section 30, and the light source driver 14 drives the lightsource provided in the illumination system 21 under the control of thecontrol section 30. The specific configuration of the projection system23 is not limited to the configuration described above and can, forexample, instead be an image formation configuration using no concavemirror or any other mirror but using a lens to project the lightmodulated by the light modulator 22 on the screen SC.

On the other hand, the image processing system primarily includes thecontrol section 30, which controls the entire projector 10, and furtherincludes a storage section 35, an operation detecting section 17, animage processing section 40, and a light modulator driver 15. Thestorage section 35 stores a control program 36 executed by the controlsection 30 and data processed by the control section 30. The imageprocessing section 40 processes an input image inputted via the imagedata input section 11 and drives the light modulator 22 on the basis ofan image signal processed by the image processing section 40 to causethe light modulator 22 to perform drawing.

The operation detecting section 17 is connected to a remote controllight receiver 18 and an operation panel 19 and detects operationreceived via the remote control light receiver 18 and the operationpanel 19.

The remote control light receiver 18 receives an infrared signaltransmitted in response to button operation from a remote control (notshown) used by the operator of the projector 10. The remote controllight receiver 18 decodes the infrared signal received from the remotecontrol, generates operation data representing the content of theoperation performed on the remote control, and outputs the operationdata to the control section 30.

The operation panel 19 is provided on an exterior enclosure of theprojector 10 and includes a variety of switches and indicator lamps. Theoperation detecting section 17 causes the indicator lamps on theoperation panel 19 to illuminate or blink as appropriate under thecontrol of the control section 30 and in accordance with the actionstate and setting state of the projector 10. When any of the switches onthe operation panel 19 is operated, operation data corresponding to theoperated switch is outputted from the operation detecting section 17 tothe control section 30.

The control section 30 outputs image data inputted from the image datainput section 11 to the image processing section 40.

The image processing section 40 includes an image input section 41, animage processing controller 42, a drawing processor 43, and a framememory 44.

The image input section 41 buffers the inputted image data.

The image processing controller 42 performs a variety of types ofconversion as appropriate, such as interlace/progressive conversion,resolution conversion, and color conversion, on the image data inputtedto the image input section 41 to generate image data having a presetformat.

The drawing processor 43 develops images on a frame basis in the framememory 44 on the basis of the image data processed by the imageprocessing controller 42.

The image data developed in the frame memory 44 is outputted to thelight modulator driver 15. The drawing processor 43 may performtrapezoidal distortion correction and pin-cushion distortion correctionon the image data developed in the frame memory 44. In this case, thedrawing processor 43 deforms the shape expressed by the image datadeveloped in the frame memory 44, for example, to a trapezoidal shapethat compensates trapezoidal distortion. The drawing processor 43further edits the images developed in the frame memory 44, specifically,adds an image of a figure or any other image to the developed images ordeletes the added image from the developed images in accordance with thecoordinates of the operation position of the pointing element 70 in anaction mode in which the projector 10 performs drawing to generate animage or update an image. An image according to operation of thepointing element 70 is thus drawn and projected on the screen SC.

The pointing element 70 includes a transmitter 74, which outputs aninfrared signal in addition to the operation switch 72. The transmitter74 includes an infrared LED or any other light emitter, a light emissioncontrol circuit, a power supply, and other components. The transmitter74 periodically transmits the infrared signal in the power-on state ofthe pointing element 70. The transmitter 74 modulates the infraredsignal in accordance with the operation state of the operation switch 72(on/off) and transmits data representing on/off of the operation switch72 in accordance with a method compliant, for example, with the IrDAstandard.

The projector 10 includes a position detecting section 50 as a detectionsection that detects a pointing action of the pointing element 70. Theposition detecting section 50 includes an imager 51, a receiver 52, animaging controller 53, a pointing element detector 54, and a coordinatecalculator 55.

The imager 51 includes an imaging optical system, an imaging device, aninterface circuit, and other components and performs imaging in theprojection direction of the projection system 23. The imaging opticalsystem of the imager 51 is so disposed that the projection system 23 andthe imaging optical system are oriented in the same direction, and theimaging optical system has a viewing angle that covers the range overwhich the projection system 23 projects an image on the screen SC.Examples of the imaging device may include a CCD and a CMOS device. Theinterface circuit reads and outputs detection values from the imagingdevice.

The imaging controller 53 causes the imager 51 to perform imaging togenerate captured image data. When the imager 51 performs imaging underthe control of the imaging controller 53, an image projected on thescreen SC and the pointing element 70 operated on the screen SC or inthe vicinity thereof is captured. In the present Embodiment 1, theimaging device of the imager 51 has sensitivity to a wavelength regionto which infrared light belongs. The imager 51 outputs captured imagedata on the infrared light collected by the imaging optical system.

The imaging section 51 may have sensitivity to the wavelength region towhich visible light belongs. Further, the imaging target is not limitedto the pointing element 70, and the operator's hand or finger may beimaged.

The receiver 52 receives and decodes the infrared signal transmitted bythe transmitter 74 and outputs received data. The received dataoutputted by the receiver 52 contains data representing the operationstate of the operation switch 72.

The pointing element detector 54 detects the position of the pointingelement 70 on the basis of the captured image data generated by theimaging controller 53. The pointing element detector 54 carries out theprocess of detecting a shape similar to the shape of the pointingelement 70 in the captured image data and cutting the image of thepointing element 70 from the captured image data to identify theposition of the pointing element 70 in the captured image data. In thisprocess, the pointing element detector 54 may identify the direction inwhich the front end of the pointing element 70 is oriented. The pointingelement detector 54 further detects the operation state of the operationswitch 72 on the basis of the received data outputted by the receiver52. The method for detecting the pointing element 70 is not limited tothe method for detecting a shape similar to the shape of the pointingelement 70, and a method for detecting light reflected off a lightcurtain is conceivable.

The method for detecting light reflected off a light curtain mayinclude, for example, outputting layer-shaped (or curtain-shaped)detection light (infrared light, for example) to the entire surface ofthe screen SC, imaging the light reflected off the pointing element 70having a portion that reflects infrared light, and detecting theposition of the pointing element 70 on the basis of the position of thereflected light in the captured image. The method for detecting lightreflected off a light curtain is not described in detail because it doesnot fall within the substance of the invention (see, for example,JP-A-2015-159523 for the method).

The coordinate calculator 55 calculates the coordinates of the positionpointed by the front end of the pointing element 70 on the screen SC,that is, the coordinates of the operation position on the basis of theposition where the pointing element 70 is located in the captured imagedata and which is detected by the pointing element detector 54.Specifically, the coordinate calculator 55 calculates the coordinatesdefined in the area where the projection section 20 projects aprojection image (projection range) on the screen SC on the basis of theoperation position in the captured image data. The coordinate calculator55 outputs data representing the calculated coordinates of the operationposition, the operation state of the operation switch 72, and otherpieces of information to the control section 30. The coordinatecalculator 55 may further output the captured image data produced by theimager 51 to the control section 30.

The control section 30 reads and executes the control program 36 storedin the storage section 35 to achieve the functions of a projectioncontroller 31, a detection controller 32, a drawn object analyzer 33,and a UI controller 34 to control the portions that form the projector10.

The control section 30 includes a CPU (central processing unit), a ROM(read only memory), a DRAM (dynamic random access memory), and othercomponents, neither of which is shown, reads a basic control programstored in the ROM into the DRAM, and causes the CPU to execute theprogram to control the projector 10. The control section 30 furtherexecutes the program stored in the storage section 35 to function as theprojection controller 31, the detection controller 32, the drawn objectanalyzer 33, and the UI controller 34.

The projection controller 31 acquires the content of operation performedby the operator on the basis of operation data inputted from theoperation detecting section 17. The projection controller 31 controlsthe image processing section 40, the light modulator driver 15, theprojection system driver 16, and the light source driver 14 inaccordance with the operation performed by the operator to cause them toproject an image on the screen SC. The projection controller 31 furthercontrols the projection system driver 16 to cause the projection system23 to perform focus adjustment, zoom adjustment, diaphragm adjustment,and other types of adjustment.

The detection controller 32 controls the position detection section 50to cause it to detect the operation position of the pointing element 70and detects the coordinates of the operation position and the operationstate of the operation switch 72.

The detection controller 32 further provides a predetermined functionrelating to drawing on the basis of the operation position of thepointing element 70 and the operation state of the operation switch 72.For example, in a case where a menu (not shown) is so projected on thescreen SC as to overlap with an image on the screen SC in response tothe operator's operation of the remote control, and the operator usesthe pointing element 70 to select a character drawing function fromoptions in the projected menu, the detection controller 32 may activatethe character drawing function that allows the operator to draw thehandwritten characters 150 on the basis of the position pointed by thepointing element 70.

The drawn object analyzer 33 analyzes characters, a figure, or any otherobject drawn on the screen SC by the operator's operation of thepointing element 70. For example, in a case where the character drawingfunction is activated, and the handwritten characters 150 are drawn asan object on the screen SC, the drawn object analyzer 33 analyzes thedrawn handwritten characters 150 to detect the drawing direction inwhich the handwritten characters 150 are drawn on the screen SC. Thedrawing direction represents the direction in which the handwrittencharacters 150 are written and coincides with the display direction inwhich the handwritten characters 150 are displayed on the screen SC.

A method for detecting the drawing direction of the handwrittencharacters 150 may include, for example, extracting an imagecorresponding to the first character of the handwritten characters 150,extracting the character from the extracted image by using a knownoptical character recognition technology (OCR technology), andcalculating the inclination of the extracted character by using rotationcorrection.

The drawn object analyzer 33 determines the drawing direction of thehandwritten characters 150 on the basis of the calculated inclination ofthe character.

The drawn object analyzer 33 may instead handle the handwrittencharacters 150 or any other figure as a schematic representation,calculate the direction vector of the schematic representation on thebasis of the group of pixels that forms the schematic representation,and determine the drawing direction on the basis of the calculateddirection vector.

Still instead, in a case where the operator holds the pointing element70 and draws an object, the drawn object analyzer 33 may determine thedrawing direction on the basis of the inclination angle of the pointingelement 70 imaged by the imager 51. In this case, information on a habitor a practice that appears when the operator holds the pointing element70 and performs drawing may be stored in the storage section 35 on anoperator basis, and the inclination angle of the pointing element 70 maybe corrected in accordance with the individual operators.

The UI controller 34 controls the position and inclination of a userinterface screen (UI screen) 160, which is projected on the screen SC,on the basis of a result of the analysis performed by the drawn objectanalyzer 33. The UI screen 160 has the function of allowingpredetermined operation to be performed on the handwritten characters150 and corresponds to a second image.

When drawing operation using the character drawing function iscompleted, the UI controller 34 calculates the angle of rotation bywhich the UI screen 160 is so inclined that the orientation of the UIscreen 160 coincides with the drawing direction of the handwrittencharacters 150 and projects the UI screen 160 in the vicinity of thehandwritten characters 150 with the UI screen 160 rotated on the basisof the calculated angle of rotation. Out of the functions of the UIcontroller 34, the function of determining the angle of rotation, whichis one of display aspects, with respect to the drawing direction of thehandwritten characters 150 corresponds to a determination section, andthe function of projecting the UI screen 160 rotated on the basis of theangle of rotation corresponds to a display control section. The displayaspect to be determined with respect to the drawing direction of thehandwritten characters 150 is not limited only to the angle of rotation.For example, in accordance with the drawing direction of the handwrittencharacters 150, the function selectable in the UI screen 160 may bechanged. As an example of the above, in a case where the drawingdirection of the handwritten characters 150 inclines with respect to thedirection parallel to the edges of the screen, the following aspect isalso conceivable as the function of the UI screen 160: The function ofresetting the inclination of the drawing direction of the handwrittencharacters 150 can be selected.

FIG. 3 is a flowchart showing the procedure of a process (informationprocessing method) carried out when the character drawing function isactivated.

When the process starts, the control section 30 draws an object on thescreen SC on the basis of the operator's moving of the pointing element70 (step S200).

The control section 30 then evaluates whether or not a predeterminedperiod has elapsed since the pointing element 70 was moved away from thescreen SC (step S202).

In a case where the predetermined period has not elapsed since thepointing element 70 was moved away from the screen SC (No in step S202),the control returns to step S200, and the character drawing functioncontinues.

On the other hand, in a case where the predetermined period has elapsedsince the pointing element 70 was moved away from the screen SC (Yes instep S202), the control section 30 calculates the drawing direction ofthe drawn object (step S204).

The control section 30 then calculates the angle of rotation of the UIscreen 160 on the basis of the calculated drawing direction (step S206).

The control section 30 then rotates the UI screen 160 in accordance withthe angle of rotation, projects the UI screen 160 in the vicinity of theobject and within a predetermined distance from the object (step S208),and terminates the series of steps.

Examples of the projection of the UI screen 160 will next be describedwith reference to FIGS. 4 to 13.

FIG. 4 shows a first UI screen 160A projected on the screen SC when adrawing action using the character drawing function is completed.

In a case where the operator operated the pointing element 70 to drawthe handwritten characters 150, and a predetermined period has thenelapsed since the operator moved the pointing element 70 away from thescreen SC, the detection controller 32 determines that the drawingaction using the character drawing function has been completed. The UIcontroller 34 receives the determination of the completion from thedetection controller 32 and projects the first UI screen 160A in thevicinity of the handwritten characters 150. In this case, the UIcontroller 34 rotates the first UI screen 160A in such a way that theorientation of the first UI screen 160A coincides with the drawingdirection of the handwritten characters 150.

The first UI screen 160A is a screen that asks the operator whether ornot the operator desires to edit the drawn handwritten characters 150.In a case where the operator desires to edit the handwritten characters150, the operator selects a Yes button 165 by pointing it with thepointing element 70. On the other hand, in a case where the operatordoes not desire to edit the handwritten characters 150, the operatorselects a No button 167 by pointing it with the pointing element 70.

FIG. 5 shows a second UI screen 160B to be projected on the screen SC inthe case where the Yes button 165 is selected in the projected first UIscreen 160A.

In this case, the UI controller 34 projects the second UI screen 160B,in place of the first UI screen 160A being projected, in the vicinity ofthe handwritten characters 150. In this case, the UI controller 34rotates the second UI screen 160B in such a way that the orientation ofthe second UI screen 160B coincides with the drawing direction of thehandwritten characters 150.

The second UI screen 160B is a screen that prompts the operator toselect the content of the editing performed on the handwrittencharacters 150. The second UI screen 160B has a range selection button162, an enlargement/reduction selection button 164, a return button 166,a color selection button 168, and a cancel button 169, each of which isa button that can issue an instruction when selected.

The range selection button 162 is a button for setting the range withinwhich the handwritten characters 150 are edited. Theenlargement/reduction button 164 is a button for enlarging or reducingthe handwritten characters 150. The return button 166 is a button thatprovides an undo function, that is, a button for canceling the action ofediting performed by using the second UI screen 160B to restore theprevious state. The color selection button 168 is a button for changingthe displayed color of the handwritten characters 150. The cancel button169 is a button for stopping editing operation using the second UIscreen 160B.

FIG. 6 shows a screen projected on the screen SC in a case where therange selection button 162 is selected in the projected second UI screen160B.

The UI controller 34 projects a range specifying window 163A, in placeof the second UI screen 160B being projected, in the vicinity of thehandwritten characters 150. In this case, the UI controller 34 projectsthe range specifying window 163A in such a way that the orientation ofthe range specifying window 163A coincides with the drawing direction ofthe handwritten characters 150 and the rectangle that forms the rangespecifying window 163A surrounds the entire handwritten characters 150.

The operator can change the range specified by the range specifyingwindow 163A by pointing and moving one edge of the rectangle that formsthe range specifying window 163A.

FIG. 7 shows an example in which the range of part of the handwrittencharacters 150 is specified by a range specifying window 163B.

That is, FIG. 7 shows that one edge of the rectangle that forms therange specifying window 163A has been so moved that part of thehandwritten characters 150 (in this case “handwritten”) is excluded fromthe range. It is noted that the operator can point and move any of thefour edges that form the rectangle.

FIG. 8 shows a case where part of the handwritten characters 150 isenlarged. The operator can specify part of the handwritten characters150 (in this case “characters”) as a range and point a vertex of therectangle to move the vertex in an arbitrary direction to specify thesize of the rectangle, that is, the factor of enlargement or reductionof the characters, as indicated by a range specifying window 163C. Afterthe pointing element 70 is moved away from the screen SC and apredetermined period elapses, the UI controller 34 senses that the rangespecifying operation has been completed and projects the second UIscreen 160B in the vicinity of the handwritten characters 150 in such away that the orientation of the second UI screen 160B coincides with thedrawing direction of the handwritten characters 150. At this point, theoperator selects the enlargement/reduction selectin button 164 in thesecond UI screen 160B.

As a result, a specified portion 154, which is part of the handwrittencharacters 150 and contained in the range specified by the rangespecifying window 163C, is enlarged in accordance with the size of therange specifying window 163C, as shown in FIG. 9. On the other hand, thesize of a non-specified portion 152, which is part of the handwrittencharacters 150 and is not specified by the range specifying window 163C,does not change.

The case where the enlargement/reduction button 164 is selected has beendescribed above. In a case where the color selection button 168 isselected, for example, a color specifying window (not shown) isprojected in place of the range specifying window 163A, and when theoperator selects a desired color in the color specifying window, thecolor of the characters in a predetermined area of the specified rangeof the handwritten characters 150 is changed to the desired color.

FIG. 10 shows Example 1 of the projection of the second UI screen 160Bin a case where a plurality of sets of handwritten characters 150A and150B are drawn by using the character drawing function. In this case, itis assumed that a first operator first draws the handwritten characters150B, and a second operator then draws the handwritten characters 150A.In this case, when a predetermined period has elapsed since thehandwritten characters 150A were drawn and the pointing element 70 wasmoved away from the screen SC, the UI controller 34 projects the secondUI screen 160B, which follows the first UI screen 160A, in the vicinityof the handwritten characters 150A in such a way that the orientation ofthe second UI screen 160B coincides with the drawing direction of thehandwritten characters 150A.

In a case where the operator draws an underline 156 below thehandwritten characters 150B, the UI controller 34 projects the second UIscreen 160B in the vicinity of the handwritten characters 150B in such away that the orientation of the second UI screen 160B coincides with thedrawing direction of the underline 156. The drawing direction maycoincide with the drawing direction of the handwritten characters 150B,which is an object closest to the drawn underline 156.

FIG. 12 shows Example 2 of the projection of the second UI screen 160Bin the case where the plurality of sets of handwritten characters 150Aand 150B are drawn by using the character drawing function.

In this case, it is assumed that the handwritten characters 150B arefirst drawn, and the handwritten characters 150A are then drawn. In thiscase, when a predetermined period has elapsed since the handwrittencharacters 150A were drawn and the pointing element 70 was moved awayfrom the screen SC, the UI controller 34 projects the second UI screen160B, which follows the first UI screen 160A, in the vicinity of thehandwritten characters 150A in such a way that the orientation of thesecond UI screen 160B coincides with the drawing direction of thehandwritten characters 150A.

In a case where the operator draws a closed area 158, which contains thehandwritten characters 150A and the handwritten characters 150B, withthe pointing element 70, the UI controller 34 draws the first UI screen160A and the second UI screen 160B on the basis of the positions ofobjects contained in the closed area 158, such as the handwrittencharacters 150A and the second handwritten characters 150B.

In this case, the second UI screen 160B may be drawn in the position ofthe center of gravity of the objects as a whole contained in the closedarea 158 or in a position in the vicinity of an object closest to thecenter of gravity of the closed area 158 (handwritten characters 150A,for example). The angle of rotation of each of the first UI screen 160Aand the second UI screen 160B may be obtained from the combination ofthe drawing directions of the objects. The angle of rotation may insteadcoincide with the drawing direction of an object closest to the centerof gravity of the objects as a whole.

According to Embodiment 1 described above, the following advantageouseffects are provided.

(1) The projector 10 projects and displays the handwritten characters150, which are drawn on the basis of a pointing action using thepointing element 70, on the screen SC, determines the drawing directionof the UI screen 160, which allows manipulation of the handwrittencharacters 150, on the basis of the drawing direction of the handwrittencharacters 150 displayed on the screen SC, and displays the UI screen160 on the screen SC on the basis of the determined drawing direction.Therefore, since the drawing direction of the UI screen 160 isdetermined on the basis of the drawing direction of the handwrittencharacters 150, the visibility of the UI screen 160 to the user who hasdrawn the handwritten characters 150 is improved, whereby the user'soperability of the UI screen 160 can be improved.

(2) Since the UI screen 160 is so displayed that the orientation thereofcoincides with the drawing direction of the handwritten characters 150,the user can readily issue a manipulation instruction via the UI screen160 in the state in which the user directly faces the handwrittencharacters 150.

(3) Since the UI screen 160 is displayed in the vicinity of thehandwritten characters 150, the visibility of the UI screen 160corresponding to the handwritten characters 150 is improved.

(4) At least one of the displayed color, enlargement, reduction, andrange specification of the handwritten characters 150 can be specifiedvia the UI screen 160.

(5) In the case where a plurality of sets of handwritten characters 150are displayed, the drawing direction of the UI screen 160 is determinedon the basis of the handwritten characters 150 pointed by a pointingaction.

Embodiment 2

Embodiment 2 of the invention will next be described. In the followingdescription, the same portions as those having already been describedhave the same reference characters and will not be described.

FIG. 14 shows the configuration of a projection system 1. In Embodiment1, the projection system 1 has the configuration including the projector10, that is, the projection system 1 is formed of a single apparatus,whereas in Embodiment 2, the projection system 1 is formed of aplurality of apparatus, such as the projector 10 and a computerapparatus 100.

The computer apparatus 100 is a kind of information processing apparatusand includes a control section 101, a storage section 110, an externalinterface (hereinafter abbreviated to “external I/F”) 109, an inputsection 106, a display processing section 107, and a monitor 108. Thecomputer apparatus 100 is assumed to be provided in the form of apersonal computer or a tablet terminal.

The control section 101 is achieved, for example, by a CPU, a ROM thatstores a basic control program and other programs executed by the CPU,and a RAM that temporarily stores the programs executed by the CPU, dataprocessed by the CPU, and other pieces of information. In Embodiment 1,the control section 30 of the projector 10 includes the detectioncontroller 32, the drawn object analyzer 33, and the UI controller 34,whereas in Embodiment 2, the control section 101 of the computerapparatus 100 includes the detection controller 32, the drawn objectanalyzer 33, and the UI controller 34.

The input section 106 detects input operation using an input deviceincluding a keyboard, a mouse, and other pointing devices. The displayprocessing section 107 displays the content of the input operationdetected by the input section 106, a result of processing performed bythe control section 101, and other pieces of information on the monitor108. The storage section 110 stores a variety of programs including acontrol program 111 executed by the control section 101, data processedby the programs, and other pieces of information in an involatilemanner.

The external I/F 109 is connected to an interface section (hereinafterabbreviated to “I/F section”) 12, with which the projector 10 isprovided, via a cable 1 a in such a way that the external I/F 109 andthe interface section 12 can communicate with each other. The externalI/F 109 has the function of the image data input section 11 described inEmbodiment 1 and the function of transmitting and receiving controlinformation and other pieces of information to and from the projector10.

The control section 101 executes the control program 111 to generatecontrol signals that control the projector 10 and transmits the controlsignals to the projector 10. The projector 10 receives the controlsignals and performs a variety of actions on the basis of the controlsignals.

The control section 101 further generates image data to be projected bythe projector 10 and transmits the generated image data to the projector10. The projector 10 receives the image data and projects an image onthe basis of the image data.

Further, the detection controller 32 receives detection informationrepresenting the position pointed by the pointing element 70 anddetected by the imager 51. The drawn object analyzer 33 analyzes anobject drawn on the screen SC by the operator's operation of thepointing element 70 on the basis of the pointing position indicated bythe detection information. The UI controller 34 instructs the projectionposition and inclination of the UI screen 160 to be projected on thescreen SC to the control section 30 of the projector 10 on the basis ofa result of the analysis performed by the drawn object analyzer 33.

In this case, the UI controller 34 calculates the angle of rotation ofthe UI screen 160 and outputs the calculated angle of rotation to theprojector 10. That is, the process of rotating the UI screen 160 iscarried out by the control section 30 of the projector 10, but theaspect described above is not necessarily employed.

For example, the UI controller 34 instead calculates the angle ofrotation of the UI screen 160, generates the UI screen 160 rotated onthe basis of the angle of rotation, and outputs the UI screen 160 to theprojector 10. The projector 10 may display the UI screen 160 outputtedfrom the computer apparatus 100. That is, it is conceivable to employ anaspect in which the projector 10 does not carry out the process ofrotating the UI screen 160 but the UI controller 34 of the computerapparatus 100 carries out the process of rotating the UI screen 160.

Embodiment 2 described above provides the following advantageous effectin addition to the advantageous effects (1) to (5) described inEmbodiment 1.

(6) Since the detection controller 32, the drawn object analyzer 33, andthe UI controller 34 are present in the computer apparatus 100, theprocess of detecting the angle of rotation of the handwritten characters150, rotating the UI screen 160 by the detected angle of rotation, andprojecting the UI screen 160 in the vicinity of the handwrittencharacters 150 can be quickly carried out.

The invention has been described above with reference to the illustratedembodiments, but the invention is not limited to the embodiments, andthe following variations are conceivable:

(1) How to display an image is not limited to the projection performedby the projector 10, and a touch panel including a liquid crystaldisplay is also conceivable.

(2) The light modulator 22 has been described with reference to theconfiguration using the three transmissive liquid crystal panelscorresponding to the RGB colors, but not limited thereto. For example, aconfiguration using three reflective liquid crystal panels may beemployed, or the combination of one liquid crystal panel and a colorwheel may be used. Still instead, the light modulator 22 may be formedof three digital mirror devices (DMDs) or may, for example, employ a DMDmethod using the combination of one digital mirror device and a colorwheel. In the case where only one liquid crystal panel or DMD is used asthe light modulator 22, a member corresponding to the light combiningsystem, such as a cross dichroic prism, is not required. Further, alight modulator 22 different from a liquid crystal panel or a DMD butcapable of modulating light emitted from a light source can be employed.

The control section 30 shown in FIG. 2 and the control sections 30 and101 shown in FIG. 14 each represent a functional configuration achievedby cooperation between hardware and software and are not eachnecessarily implemented in a specific form. Therefore, hardwarecorresponding to each of the functional portions is not necessarilyimplemented, and a single processor that executes a program can, ofcourse, achieve the functions of the plurality of functional portions.Further, in the embodiments described above, part of the functionsachieved by software in the embodiments described above may be achievedby hardware, or part of the functions achieved by hardware may beachieved by software.

What is claimed is:
 1. A display apparatus comprising: a projector thatdetects a pointing action; a display that displays a first image basedon the pointing action; a controller that detects a display direction ofthe first image in the display; and a processor that functions as: adetermination section that determines, based on the display direction ofthe first image in the display, a display aspect of a second image insuch a way that a display direction of the second image coincides withthe display direction of the first image, the second image beingdifferent from the first image and including options, the optionsallowing manipulation of the first image; and a display control sectionthat causes the display to display, along with the first image, thesecond image based on the display aspect determined by the determinationsection with the options displayed in the second image.
 2. The displayapparatus according to claim 1, wherein the display control sectionrotates the second image based on the display aspect and displays therotated second image in a vicinity of the first image.
 3. The displayapparatus according to claim 1, wherein at least one of a displayedcolor, enlargement, reduction, and range specification of the secondimage is capable of being specified with respect to the first image. 4.The display apparatus according to claim 1, wherein in a case where thefirst image is formed of a plurality of first images and the pluralityof first images are displayed in the display, the determination sectiondetermines the display aspect of the second image based on the firstimage pointed by the pointing action.
 5. The display apparatus accordingto claim 4, wherein in a case where the pointing action points one ofthe plurality of first images, the determination section determines thedisplay aspect of the second image in such a way that an orientation ofthe second image coincides with the display direction of the pointedfirst image.
 6. The display apparatus according to claim 4, wherein in acase where the pointing action points the plurality of first images, thedetermination section determines the display aspect of the second imagebased on directions of the plurality of first images.
 7. An informationprocessing apparatus that processes a first image displayed based on apointing action, the information processing apparatus comprising: aprocessor that functions as: a determination section that determines,based on a detected display direction of the displayed first image, adisplay aspect of a second image in such a way that a display directionof the second image coincides with the display direction of the firstimage, the second image being different from the first image andincluding options, the options allowing manipulation of the first image;and a display control section that displays, along with the first image,the second image based on the display aspect determined by thedetermination section, with the options displayed in the second image.8. An information processing method comprising: detecting a pointingaction; displaying a first image on a display based on the pointingaction; detecting a display direction of the first image on the display;determining, based on the display direction of the displayed firstimage, a display aspect of a second image in such a way that a displaydirection of the second image coincides with the display direction ofthe first image, the second image being different from the first imageand including options, the options allowing manipulation of the firstimage; and displaying, on the display and along with the first image,the second image based on the determined display aspect, with theoptions displayed in the second image.